Exception handling in automated data reading systems

ABSTRACT

Disclosed are systems and methods for unassisted (i.e., customer controlled) exception handling in an automated data reader having a read zone. A first imager obtains a first image of an exception item in response to an exception generated in a read zone. An exception handling station receives the exception item, and a second imager located at the exception handling station obtains a second image of the exception item. An image processor receives the images, identities visual object recognition features from each image, and compares the features to determine whether the first and second images represent the same exception item. If so, the exception is cleared, and the item is added to a transaction list.

RELATED APPLICATION

This application claims the benefit under 35 U.S.C. §119(e) of U.S.Provisional Patent Application No. 61/735,517, filed Jul. 19, 2012,which is hereby incorporated by reference in its entirety.

BACKGROUND INFORMATION

The field of this disclosure relates generally to systems and methodsfor automated data reading and, more particularly, to exception handlingsystems and to methods of exception handling for automated checkoutsystems.

Optical code readers acquire data from 1-D and 2-D optical codes or fromother types of identifying indicia or symbols, such as biometricfeatures. Two types of optical code readers used to acquire dataassociated with an optical code are laser scanners and imager-basedoptical code readers—the latter are also referred to as imaging readers.Both laser scanners and imaging readers may be referred to moregenerally as scanners, data readers, or simply, readers. Therefore, forpurposes of this disclosure, the terms scan and read are usedinterchangeably to connote acquiring information associated with opticalcodes. Likewise, the terms scanner and reader are used interchangeablyto connote devices that acquire data associated with optical codes,other symbols, or electromagnetic fields (e.g., radio-frequencyidentification or near field communication).

Optical codes are typically placed on items and read by data readers tohelp track item movement in industrial or shipping facilities, or tofacilitate sales and monitor inventory at retail establishments. Theoptical codes are placed on or associated with items, packages,containers, or other objects and read by the data reader when the itemsbearing the optical codes are within a read zone during a data-readingoperation. For example, in retail stores, data readers are placed atcheckstands or are built into a checkstand counter and generally haveone or more read volumes (scan volumes) that collectively establish aread zone in which optical codes may be successfully read.

Data readers that read the information encoded in optical codes may begenerally classified into one of three types: manual readers,semi-automatic, and automated readers. With manual or semi-automaticreaders (e.g., a hand-held type reader, or a fixed-position reader), ahuman operator positions an item relative to the read zone to read theoptical code associated with the item. In an automated reader (e.g., aportal or tunnel scanner), a conveyor automatically positions the itemrelative to the read zone, and transports the item through the read zoneso that the data reader can automatically read the optical code borne bythe item.

Any of these three types of readers can be used in either assisted orself-checkout processes. In an assisted checkout process, a customerplaces items on a counter, deck, or conveyor of a checkstand; the itemsare transported to a checkout clerk (checker); and the checker thentakes an item and moves it into or through the read zone of the datareader. Accordingly, the checker typically locates an optical code on alabel of the item, and then holds the label in a particular orientationto obtain a successful read of the optical code as it is moved throughthe read zone. In a self-checkout process, a customer (or otheroperator) operates the data reader, unassisted by a checker or otherdedicated attendant. In other words, the customer acts as the checkerand oversees the data-reading operations during the self-checkoutprocess.

SUMMARY OF THE DISCLOSURE

Systems and methods are disclosed that provide for unassisted exceptionhandling at an exception handling station of an automated data reader.According to one embodiment, a conveying system transports items bearingidentification indicia through a read zone of an automated data reader.The data reader reads the identification indicia for a successfullyidentified item, and produces an exception associated with anunidentified item (i.e., an item that is not successfully identified).An unidentified item that has passed through the read zone of anautomated data reader and has an exception associated with it is alsoknown as an exception item. A first imager is located proximal an outletof the read zone and, in response to the exception, obtains first imagedata representing the exception item. An exception handling stationreceives the item that has the exception, and a second imager at thestation obtains second image data representing this item. An imageprocessor receives the first and second image data and extractscorresponding first and second visual recognition features, and comparesthe first and second visual recognition features to confirm whether theunidentified item is represented in both the first and second imagedata. The second image data optionally includes identification indiciaused to identify the exception item and to clear the exception. Anoptional display located in the exception handling station, for example,alerts a user of the exception, provides instruction for clearing theexception, or displays the image data, which may include video data.

In some embodiments, the identification indicia includes an optical codeor visual recognition features such as scale-invariant featuretransformation (SIFT) features. Additionally, the visual recognitionfeatures may also be SIFT features, or other object recognition featuressuitable for various object recognition techniques.

Additional aspects and advantages will be apparent from the followingdetailed description of preferred embodiments, which proceeds withreference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are described below with reference to accompanying drawings;however, the accompanying drawings depict only certain embodiments andare therefore not intended to limit the teachings of this disclosure.

FIG. 1 is an isometric view of an automated data reader embodied as anautomated checkout system including a conveying system with some itemsloaded on an input conveyor and other items entering a read zone, asviewed from a customer's point of ingress.

FIG. 2 is a block diagram of an automated data reading system, accordingto one embodiment.

FIGS. 3 and 4 are isometric views of the automated checkout system ofFIG. 1 as viewed from a customer's point of egress, illustrating,respectively, a first imager operative for obtaining a first image of anitem having an associated exception and a second imager operative forobtaining a second image for clearing the associated exception at anexception handing station.

FIGS. 5A and 5B are renderings of, respectively, the first imageacquired by a first imager and the second image acquired by the secondimager at the exception handling station of FIGS. 3 and 4.

FIG. 6 is a flowchart showing a method of automated data reading,according to one embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

In both assisted or self-checkout processes, the present inventor hasrecognized that problems can arise due to a misread or a non-read of anoptical code (also referred to as an exception), slowing the checkoutprocess. For example, exceptions can result from any of the following: adamaged optical code, an optical code that is obscured or occluded fromview of the data reader, misalignment of the optical code (e.g.,misaligned barcode lines), inadvertent movement of the optical code awayfrom the read zone, identifying information for an imaged optical codethat is unavailable in an inventory database, a mismatch between anoptical code and other detected visual characteristics of the item(e.g., size, shape, or brand labeling), or other problems causingexceptions. The likelihood or frequency of an exception is exacerbatedin self-checkout systems using manual or semi-automatic readers becauseoperators (i.e., a customer) may not be familiar with the self-checkoutprocesses or readers, or they may have difficulty in locating andpositioning optical codes in a read zone for producing successful datareads. Likewise, prior automated scanners would generate exceptionsbecause these systems occasionally failed to achieve successful scans onthe first pass of an item through a scan zone due to the wide variationsin product sizes, irregularities of packaging shapes, differinglocations of barcodes, and due to larger items overshadowing, crowding,or concealing neighboring items.

Once exceptions are identified, the exceptions are typically resolved ina process referred to as assisted exception clearing. Assisted exceptionclearing generally entails an attendant rescanning the item—often withhandheld scanners—in order to obtain data associated with the barcodeson packages that cause exceptions. Although assisted exception clearingslows down the checkout process when checkers or other attendants maynot be available to rescan items, checkers are used because customersmay not have a handheld scanner, do not know how to operate a handheldscanner, attempt to clear exceptions from packages that did not generatethe exception, or they may have various other difficulties clearing theexception.

Shown from a point of ingress of a customer 2, FIG. 1 is an isometricview of an automated data reading system 4 (or simply, system 4). Thesystem 4 includes an automated data reader 5 (or simply, reader 5)installed on an automated checkout system 6 including an inlet end 8, anoutlet end 10, and a conveying system 12 for conveying items 14 (e.g.,merchandise) through a three-dimensional volume (referred to as a readzone 16) of the reader 5. The read zone 16 is defined by fields of view36 generated by imagers 40 (or other suitable data capture devices) ofthe reader 5. The reader 5 has an inlet housing section 50 and an outlethousing section 52 that encompass the imagers 40 and form correspondingarches 54 over the conveying system 12. Details of the reader 5 and thecorresponding arches 54 are further described in U.S. Patent ApplicationNo. 61/435,777, titled, “Tunnel Scanner for Automated Checkout;” and inU.S. patent application Ser. No. 13/357,356, titled, “Tunnel or PortalScanner and Method of Scanning for Automated Checkout,” each of which ishereby fully incorporated by reference. Although the reader 5 isillustrated with inlet and outlet housing sections 50, 52 including anopen space between each other, in other embodiments a portal reader maybe embodied in an elongate tunnel formed over the conveying system 12,or a single housing section or arch.

The reader 5, which may include an optical code reader, is operable toobtain image, dimensional, and positional data representing the items 14transported by the conveying system 12 through the read zone 16. Toautomatically move the items 14 along a transport path 56 through theread zone 16, the conveying system 12 may include one or more suitabletypes of mechanical transport systems. To track the items 14, the system4 includes, for example, conveyor-speed encoders to allow for thecalculation of dead-reckoning positional information, and opticaldetectors 58 on the arches 54 to provide dimensional information of theitems 14. Thus, the system 4 is configured to automatically position andtrack the items 14 within and through the read zone 16.

Once the items 14 are positioned in the read zone 16, the reader 5 readsoptical codes 60 or other identifying indicia (e.g., visuallyrecognizable features) borne by the items 14. The reader 5 andassociated subsystems described below with reference to FIG. 2,determine whether identifying indicia are present in the images, decodethat information, and identify the tracked items based on theinformation. Each of the successfully identified items is optionallyadded to an item transaction list 64, which is presented on an inletdisplay 66, for example.

In a successful identification operation, the reader 5 reads an opticalcode and confidently associates it to one item. For example, the reader5 reads an optical code 70 and associates it with a canister 72. Thedimensions and position of the canister 72 are detected so that thecanister 72 may be tracked while being transported through the read zone16, thereby producing a single identification and corresponding entry onthe transaction list 64. Accordingly, in a successful read, the canister72 does not have other optical codes associated with it, or any otherproblems that would otherwise cause an exception. For example, one typeof exception, called a no-code exception, is an event characterized bythe system 4 tracking an item while it is transported through the readzone 16, but no optical code is read by the data reader 5. Another typeof exception, called a no-item or phantom-read exception, ischaracterized by an event in which an optical code is read, but thesystem 4 does not detect that an item has passed through the read zone16. Other types of exceptions applicable to systems and methods of thisdisclosure are described in U.S. patent application Ser. No. 13/357,459,titled, “Exception Detection and Handling in Automated Optical CodeReading Systems,” which is hereby fully incorporated by reference.

In the event of an exception, the system 4 includes an exceptionhandling station 80 that includes an outlet display 82 and an imagingreader 84 (e.g., a camera, or other imager) affixed to the top of thedisplay 82 or other suitable location. The exception handling station 80is located proximal the outlet housing section 52, downstream along thepath 56 of the conveying system 12 at the outlet end 10 configured toreceive items from the data reader 5 (e.g., a bagging area). Thus, auser (e.g., the customer 2) can readily observe and subsequently clearthe exception without assistance of a clerk. In other words, theexception handling station 80 provides for unassisted exceptionhandling. An overview of an exception handling system 80 is describedbelow, including an example scenario of unassisted exception handlingusing the exception station 80 described with respect to FIGS. 3-4, 5A,and 5B. Methods of using the exception handling station are alsodescribed, in particular with respect to FIG. 6.

In some embodiments, exception handling stations (also referred to asprocessing stations or systems) may have smaller areas than that ofstation 80. Additionally, in other embodiments, processing stations maybe located remotely away from the reader 5.

FIG. 2 is a block diagram of an automated data reading system 100 (orsimply, system 100), according to one embodiment. The system 100 and itsassociated subsystems may include decoders (e.g., software algorithms,hardware constructs) to decode various types of identifying indicia thatinclude different types of optical codes, such as one-dimensional (e.g.,linear) codes (e.g., UPC, codabar, code 25, code 39, code 93, code 128,code 11, EAN8, EAN13, plessey, POSTNET), stacked linear codes (e.g., GS1Databar, PDF417), and two-dimensional (e.g., matrix) codes (e.g., azteccode, maxicode, QR code, high-capacity color barcode, data matrix). Insome embodiments, the identifying indicia may include visual recognitionfeatures, such as geometric point features or other scale-invariantfeature transformation (SIFT) features.

To obtain dimensional information from items (see e.g., the items 14 ofFIG. 1) that are transported by a conveying system 105, such asconveying system 12, the system 100 includes an item measurement system115 (see e.g., the optical detectors 58, of FIG. 1). The itemmeasurement system 115 generates volumetric-model or dimensional datathat represent items as three-dimensional models. The three-dimensionalmodels are used to track each item as it is transported by the conveyingsystem 105 through a read zone of a data reading system 120.

The data reading system 120 is also operable to generate projection datafor optical codes represented in the images it captures. The projectiondata represent back projection rays that project into the read zone ofthe data reading system 120. These back projection rays are associatedwith locations of the representations of the optical codes in theimages, and they facilitate association of tracked items with an opticalcode. For example, the system 100 includes an optical code intersectionsystem 125 that is configured to receive the model data from the itemmeasurement system 115 and the projection data from data reading system120. The optical code intersection system 125 then uses this receivedinformation to determine whether the back projection rays generated fordecoded optical codes intersect with the three-dimensional models, forpurposes of identifying exceptions.

The system 100 includes an exception identification system 130 thatcommunicates with the optical code intersection system 125. Theexception identification system 130 is configured to determine whetheroptical codes read by the data reading system 120 are associated withthree-dimensional models generated by the item measurement system 115.In one example, the exception identification system 130 determines thatthe optical codes are associated with the three-dimensional models basedon intersection determinations made by the optical code intersectionsystem 125. From the associations (or lack of associations) of theoptical codes and three-dimensional models, the exception identificationsystem 130 may determine whether exceptions occur. For example, if anitem passes through the read zone of the data reading system 120 and theitem measurement system 115 generates a three-dimensional model of theitem, but no optical code is associated with the three dimensional model(e.g., no back projection ray of an optical code intersects thethree-dimensional model), the exception identification system 130identifies this event as a no-code exception. The exceptionidentification system 130 is also operable to classify and categorizeexceptions by types and subtypes and to generate exception categoryidentification information indicative of the exceptions' types and/orsubtypes. Additional details of the exception identification system 130are described in the aforementioned '459 application.

The system 100 includes an exception handling system 135 incommunication with the exception identification system 130. Theexception handling system 135 determines in what manner to handle orresolve an exception identified by exception identification system 130based on the exception's type. To this end, the exception categoryidentification information generated by the exception identificationsystem 130 is communicated to the exception handling system 135. Theexception handling system 135 is operable to determine that an exceptionshould be resolved in one of multiple ways. For example, the exceptionhandling system 135 may determine that an exception is to beautomatically resolved (e.g., ignoring the exception) or manuallyresolved by an operator (e.g., the customer 2). The exception handlingsystem 135 may communicate with an optional storage device 140 thatstores various types of information associated with exceptions,including images of exception items. One embodiment of the exceptionhandling system 135 is described in greater detail below with referenceto FIGS. 3-4.

The system 100 may also include an optional exception item annotationsystem 145 that is operable to generate annotated image datacorresponding to visual representations of exceptions to enable acustomer to readily identify which items transported through the readzone have associated exceptions. The annotated image data generated bythe item annotation system 145 are communicated to a display 150, suchas the outlet display 82 (FIG. 1), which displays the visualrepresentations of the exceptions. The item annotation system 145 isdescribed in greater detail in the aforementioned '459 application.Example renderings of annotated image data are shown in FIGS. 5A and 5B,discussed below.

The system 100 and its subsystems may include computing devices, such asprocessors, and associated software or hardware constructs, and/ormemory to carry out certain functions and methods. The computing devicesmay be embodied in a single central processing unit, or may bedistributed such that a system has its own dedicated processor.Moreover, some embodiments of subsystems may be provided as a computerprogram product including a machine-readable storage medium havingstored thereon instructions (in compressed or uncompressed form) thatmay be used to program a computer (or other electronic device) toperform processes or methods described herein. The machine-readablestorage medium may include, but is not limited to, hard drives, floppydiskettes, optical disks, CD-ROMs, DVDs, read-only memories (ROMs),random access memories (RAMs), EPROMs, EEPROMs, flash memory, magneticor optical cards, solid-state memory devices, or other types ofmedia/machine-readable medium suitable for storing electronicinstructions. Further, embodiments may also be provided as a computerprogram product including a machine-readable signal (in compressed oruncompressed form). Examples of machine-readable signals, whethermodulated using a carrier or not, include, but are not limited to,signals that a computer system or machine hosting or running a computerprogram can be configured to access, including signals downloadedthrough the Internet or other networks. For example, distribution ofsoftware may be via CD-ROM or via Internet download.

FIGS. 3-4 are isometric views of the system 4, as viewed from a point ofegress of the customer 2. FIG. 3 shows a first imager 180 obtainingfirst image data 182, or simply, image 182 (see, e.g., image 182 of FIG.5A) of an item 188 having an associated exception (exception item 188).FIG. 4 shows the second imager 84 obtaining second image data 190, orsimply, image 190 (see e.g., image 190 of FIG. 5B) for clearing theassociated exception at the exception handing station 80.

As shown in FIG. 3, the first imager 180 is located on the outlethousing section 52 of the data reader 5, and it is positioned to producea field of view 200 that encompasses items as they exit the read zone16. As exception items enter the field of view 200, the imager 180receives a signal to acquire the image 182 (FIG. 5A), which representsthe exception item 188 in the read zone 16, or other items that have anassociated exception in the read zone 16. The image 182 may also includevideo data, and it may be presented on the display 82 that is viewableby the customer 2 standing near the exception handling station 80. Animage processor 202 included in the system 4 processes the image 182 toextract visual recognition features (e.g., SIFT features) from the image182, which are used to clear the exception securely and withoutassistance of an attendant, as described below.

As depicted in FIG. 4, when the exception item 188 reaches the exceptionhandling station 80, the display 82 presents instructions 210 to thecustomer 2 that instruct the customer 2 to position the exception item188 within a field of view 212 of the imager 84, and optionally, toposition the exception item 188 with its optical code 214 visible to theimager 84. The exception handling station 80 then acquires the image 190(FIG. 5B) by prompting the customer 2 to manually initiate an imagecapture. In some embodiments, the image capture may be automaticallyinitiated without initiation by the customer. For example, the imagecapture may be initiated as part of a continuous image capture sequence,initiated after some delay, or initiated when motion is detected in thefield of view 212 for a selected duration. In some embodiments, theimage capture of the imager 84 is continuous; the exception handlingstation 80 continuously searches for SIFT features that match featuresfrom the image 182 (FIG. 5A) while also concurrently searching foroptical codes in order to passively clear exceptions, as describedbelow.

Once the image 190 is obtained, the image processor 202 (which may bepart of the reader 5, or located remotely), or another image processor220 located in the exception handling station 80, extracts visualrecognition features from the image 190. The processor 202 or 220compares the two sets of features to determine whether the featuresmatch and thereby indicate that the same exception item 188 is shown inboth the images 182 and 190. In some embodiments, a match of visualrecognition features is determined when a sufficient number of visualrecognition features in first image data are the same or are similar tovisual recognition features in second image data, or are arranged in asimilar geometric pattern in two different images. Additional details ofextracting and comparing features are described below with respect toFIG. 6. Feature comparison and object recognition are also described inU.S. Pat. No. 8,196,822, titled, “Self Checkout with VisualRecognition,” which is hereby fully incorporated by reference. One suchknown method of object recognition is described in U.S. Pat. No.6,711,293, titled, “Method and Apparatus for Identifying Scale InvariantFeatures in an Image and Use of Same for Locating an Object in anImage.”

If the exception item 188 does not appear in both the images 182 and190, the display 82 produces a notification that alerts the customer 2of the problem, and requests the customer 2 attempt to clear theexception, or optionally informs a checker 230 that assistance isneeded. However, if the exception item 188 is determined to be in boththe images 182 and 190, the data reader 5 (or a separate decode module,which may be at a point of sale, or performed by the processor 202 or220) decodes the optical code 214 to identify the exception item 188,thereby clearing the exception and placing the exception item 188 on thetransaction list 64.

The exception handling station 80 clears the exception by comparing SIFTfeatures (or other visual recognition features) between the images 182and 190, verifying the exception item 188 appears in both, andconfirming that decoded information (e.g., decoded optical code 214) inthe image 190 is associated with the exception item 188 shown in theimage 182. In another embodiment, if it is determined the exception item188 appears in both the images 182 and 190, the visual features from theimage 182 or 190 are compared to a visual recognition feature databaseof known items in order to match the visual features from the images 182or 190 to the visual features of a known item in the database, therebyidentifying the exception item 188 and clearing the exception.Additional details of feature comparison and object recognition aredescribed in U.S. Patent Application Publication No. 2011/0286628,titled, “Systems and Methods for Object Recognition Using a LargeDatabase,” which is hereby fully incorporated by reference.

According to one embodiment, decoded information from image 190 isassociated with the exception item 188 in the image 182 based on acontinuous sequence of image frames showing the exception item 188traveling from the read zone 16 to the exception handling station 80.The fields of view 200 and 212 may also partly overlap so that matchingSIFT features among the sequence may confirm the exception item 188 iscontinuously present in at least one of the fields of view 200 or 212while the exception item 188 travels from the read zone 16 to theexception handling station 80. Thus, the matching features in thesequence establish that decoded information from the image 190 is alsoassociated with the exception item 188 shown in the image 182.

In some embodiments, an exception handling station may optionallyinclude a pivoting gate that allows items without exceptions to bypassthe exception handling station and arrive unimpeded at the outlet end 10(e.g., a bagging area). Other embodiments may include an exceptionhanding station that is located along a different transport path, e.g.,a path transverse to the path 56, or may include a separate conveyingsystem spaced apart from a bagging area.

As shown in FIGS. 5A and 5B, the exception item 188 may be annotated inthe images 182 or 190 with annotated edges 240 that contrast the visualrepresentation of the exception item 188 with the image background andwith other superfluous details of the images 182 and 190. Annotatedfeatures, such as edges 240, text, arrows, patterns, or otherannotations differentiate exception items from other successfullyidentified items, and when the images 182 or 190 are displayed on thedisplay 82, the customer 2 can readily visually isolate the exceptionitem 188 from other successfully identified items.

FIG. 6 is a flowchart showing a method 300 of automated data reading,according to one embodiment. The steps of the method 300 are presentedaccording to one example order, but the order of the steps in the method300 may be rearranged when appropriate. Additionally, some of the stepsmay be performed in parallel (e.g., at the same time), and withoutintervention of a checker or other attendant. The method 300 includesthe following steps.

Step 310 includes transporting or transiting items through a read zoneof a data reader configured to acquire identification information of asuccessfully identified item transported through the read zone. Asdescribed above, the items 14 are transported via the conveying system12 through the read zone 16 of the reader 5. The reader 5 is configuredto acquire images of optical codes borne by the items 14, and to decodethe optical codes during a successful data read operation.

Step 320 includes identifying an exception for an unidentified itemtransported through the read zone. For example, the exceptionidentification system 130 identifies exceptions for varioussituations—including for no-code or phantom-read situations—and theexception is associated with an exception item, such as the exceptionitem 188 (FIGS. 3-4, 5A, and 5B).

Step 330 includes obtaining first image data representing theunidentified item. As noted above, the first image 182 is obtained asthe exception item 188 exits the read zone 16; however, the first image182 may also be acquired from imagers 40 forming the read zone 16 whilethe items 14 are within the read zone 16 or in any other suitable mannerthat allows for an exception image to be captured.

Step 340 includes transiting items from the read zone to an exceptionhandling station. In the system 4, for example, the items 14 aretransited by the conveying system 12 from the read zone 16 to theexception handling station 80.

At the station 80, step 350 includes obtaining second image datarepresenting the unidentified item. An example of the second image datais the image 190 of FIG. 5B. The image 190 may be obtained automaticallyas exception items enter the station 80, and may also be obtained inresponse to the customer 2 manually initiating the exception clearingoperation.

Step 360 includes extracting from the first and second image data,corresponding first and second visual recognition features. The firstand second visual features are, for example, first and second sets ofgeometric point features, but other visual recognition features arepossible. The features are extracted using the image processor 202, theimage processor 220, both, or another suitable computing deviceconfigured as a feature extractor. The feature extractor extracts fromeach of the images 182 and 190, geometric point features that mayinclude one or more of different types of features such as, but notlimited to, scale-invariant feature transformation (SIFT) features,described in the '293 patent, noted above; speeded up robust features(SURF), described in Herbert Bay et al., “SURF: Speeded Up RobustFeatures,” Computer Vision and Image Understanding (CVIU), Vol. 110, No.3, pp. 346-359 (2008); gradient location and orientation histogram(GLOH) features, described in Krystian Mikolajczyk & Cordelia Schmid, “Aperformance evaluation of local descriptors,” IEEE Transactions onPattern Analysis & Machine Intelligence, No. 10, Vol. 27, pp. 1615-1630(2005); DAISY features, described in Engin Tola et al., “DAISY: AnEfficient Dense Descriptor Applied to Wide Baseline Stereo,” IEEETransactions on Pattern Analysis and Machine Intelligence, (2009); andany other types of features that encode the local appearance of theexception item 188 (e.g., features that produce similar resultsirrespective of how the image of the exception item 188 was captured,irrespective of variations in illumination, scale, position andorientation).

The feature extractor produces for each of the images 182 and 190,feature data representing a feature model of the exception item 188. Afeature model corresponds to a collection of features that are derivedfrom the images 182 and 190. Each feature model may include differenttypes of information associated with a feature and with the exceptionitem 188, such as an identifier to identify that the feature isassociated with a specific image or item; the X and Y positioncoordinates, scale, and orientation of the feature; and amulti-dimensional feature descriptor of each feature.

Step 370 includes comparing the first and second visual recognitionfeatures to confirm whether the unidentified item is represented in boththe first and second image data and to ensure that the unidentified itemis the same item that caused an exception. For example, the SIFTalgorithm is used to verify that the features in the image 182 match thefeatures from the image 190 presented at the exception handling station80. With the SIFT algorithm, extracted feature descriptors of one imageare compared to the extracted feature descriptors in another image tofind nearest neighbors. Two features match when the Euclidian distancebetween their respective SIFT feature descriptors is below somethreshold. These matching features, referred to here as nearestneighbors, may be identified in any number of ways including a linearsearch (brute force search). In other embodiments, a pattern recognitionmodule identifies a nearest neighbor using a Best-Bin-First search inwhich the vector components of a feature descriptor are used to search abinary tree composed from each of the feature descriptors of the otherimages to be searched. Although the Best-Bin-First search is generallyless accurate than the linear search, the Best-Bin-First search providessubstantially the same results with significant computational savings.

With the features common to the images identified, the image processordetermines the geometric consistency between the combinations ofmatching features. In one embodiment, a combination of features(referred to as feature patterns) is aligned using an affinetransformation, which maps the coordinates of features of one image tothe coordinates of the corresponding features in another image. If thefeature patterns are associated with the same underlying item, thefeature descriptors characterizing the item will geometrically alignwith small difference in the respective feature coordinates. The degreeto which feature patterns match (or fail to match) can be quantified interms of a residual error computed for each affine transform comparison.A small error signifies a close alignment between the feature patternswhich may confirm that the same underlying item is being depicted in thetwo images. In contrast, a large error generally indicates that thefeature patterns do not align, although common feature descriptors matchindividually by coincidence.

The exception may be cleared when there are a sufficient number ofmatching features between the images 182 and 190, and when informationis decoded from the image data so that the exception item 188 isconfirmed to have caused the original exception. The exception iscleared by obtaining identifying information, such as an optical code,from the first or second image data, and confirming based on matchingSIFT features that the identifying information is associated with theexception item 188 represented in the image 182. Once the exception iscleared, the exception item 188 may be added to the transaction list 64.

The aforementioned embodiments of a data reader are described in aretail setting that should not be considered limiting. Other uses fordata readers with the characteristics and features as described may bepossible, for example, industrial locations such as a parceldistribution (e.g., postal) station are contemplated and within thescope of this disclosure. Furthermore, though examples are providedprimarily with respect to an automated data reader, the systems andmethods may be employed in self-checkout systems using manual orsemi-automatic data readers. Finally, skilled persons should understandthat many changes may be made to the details of the above-describedembodiments, without departing from the underlying principles of thisdisclosure. Thus, the scope of the present invention should bedetermined only by the following claims.

1. An automated data reading system comprising: a data reader defining aread zone and configured to acquire identification information of anitem transported through the read zone; a processor configured todetermine whether the item has been successfully identified; a firstimager to obtain, in response the item not being successfullyidentified, first image data representing the item positioned near anoutlet of the read zone; an exception handling station configured toreceive the item, the exception handling station including a secondimager to obtain second image data representing the item positioned atthe exception handling station; and an image processor configured toobtain from the first and second image data corresponding first andsecond visual object recognition features representing a visual propertyof the item, and further configured to compare the first and secondvisual object recognition features to determine whether the item appearsin both of the first and second image data.
 2. An automated data readingsystem according to claim 1, in which the image processor is furtherconfigured to acquire from the second image data, exception-handlingidentification information of the item, and to identify the item basedon the exception-handling identification information.
 3. An automateddata reading system according to claim 1, in which the identificationinformation is encoded in a barcode.
 4. An automated data reading systemaccording to claim 1, in which the identification information comprisesscale-invariant feature transformation features.
 5. An automated datareading system according to claim 1, further comprising: a housingencompassing the data reader, the housing including an inlet side and anoutlet side, wherein the first imager is located within the housing andpositioned to project a field of view into the read zone.
 6. Anautomated data reading system according to claim 1, further comprising:a housing encompassing the data reader, the housing including an inletside and an outlet side, wherein the first imager is located above thehousing and positioned to project a field of view encompassing a portionof the outlet side.
 7. An automated data reading system according toclaim 1, in which the first imager comprises an imager of the datareader that is configured to acquire the identification information inresponse to an initial data-reading operation.
 8. An automated datareading system according to claim 1, in which the exception handlingstation includes a user display configured to present the first imagedata.
 9. An automated data reading system according to claim 8, in whichthe user display is configured to present user instructions directing auser to obtain the second image data with the second imager.
 10. Anautomated data reading system according to claim 1, in which the firstimage data comprises video data.
 11. An automated data reading systemaccording to claim 1, in which the visual object recognition featurescomprise scale-invariant feature transformation features.
 12. Anautomated data reading system according to claim 1, further comprising atransport path, and in which the exception handling station is locatedalong the transport path.
 13. An automated data reading system accordingto claim 1, further comprising: a conveying system configured totransport items through the read zone; an inlet portion; and an outletportion including the exception handling station, in which the conveyingsystem is operative to transport items from the inlet end, through theread zone, and to the outlet end.
 14. An automated data reading systemaccording to claim 1, further comprising: an annotation system operableto annotate the first or second image data to indicate an itemrepresented in the first or second image data has not been successfullyidentified.
 15. A method of automated data reading, the methodcomprising: obtaining first image data representing an unidentified itemtransported through a read zone of a data reader; moving items from theread zone to an exception processing station; obtaining at the exceptionprocessing station second image data representing the unidentified item;extracting from the first and second image data, corresponding first andsecond visual object recognition features representing a visual propertyof the unidentified item; and comparing the first and second visualobject recognition features to confirm whether the unidentified item isrepresented in both the first and second image data.
 16. A method ofautomated data reading according to claim 15, further comprising:identifying, based on the second image data, the unidentified item toestablish a subsequently identified item.
 17. A method of automated datareading according to claim 16, further comprising: adding thesubsequently identified item to a transaction list.
 18. A method ofautomated data reading according to claim 16, further comprising:generating an exception in response to the unidentified item beingtransported through the read zone; and clearing the exception inresponse to establishing the subsequently identified item.
 19. A methodof automated data reading according to claim 15, further comprising:alerting a user in response to the unidentified item being transportedthrough the read zone by displaying the first image data on a userdisplay located at the exception processing station.
 20. A method ofautomated data reading according to claim 15, in which the second imagedata includes a representation of an optical code, the method furthercomprising: attempting to decode the optical code to identify theunidentified item from the second image data.
 21. A method of automateddata reading according to claim 20, further comprising: displaying on auser display located at the exception processing station a notificationof whether the optical code was decoded.
 22. A method of automated datareading according to claim 15, further comprising: displaying the secondimage data on a user display located at the exception processingstation.
 23. A method of automated data reading according to claim 15,further comprising: annotating the first image data to generateannotated image data corresponding to visual representations thatindicate the unidentified item being transported through the read zone.